test: A2A ITK interoperability harness + baseline (no SDK changes)#35
test: A2A ITK interoperability harness + baseline (no SDK changes)#35zeroasterisk wants to merge 1 commit into
Conversation
Adds a test-harness that drives the unmodified A2A Elixir SDK against the
official A2A Interoperability Test Kit (ITK), plus an honest capability/gap
baseline report. This is a measuring-stick PR — no changes to lib/.
Harness:
- test/support/itk/instruction.ex ITK Instruction protobuf codec
- test/support/itk/agent.ex JSON-RPC handler / instruction interpreter
- test/itk/server.exs standalone Bandit server (v0.3 card, JSON-RPC, SSE)
- test/itk/{instruction,agent}_test.exs + fixtures
Report (docs/ITK_BASELINE.md) documents, with reproducible evidence:
- WORKS: v0.3 agent card, proto decode, interpreter, non-streaming message/send (404 tests green)
- GAPS: JSON-RPC enum encoding (ROLE_AGENT/TASK_STATE_* vs v0.3 agent/completed),
SSE streaming event envelopes, v1.0 vs v0.3 card shape
- Prioritized v1.0 gap list seeding future (separate) SDK work
Reference client: a2a-sdk 0.3.24.
|
The plan is: Land this PR with dustin for v0.3 and 1.0. then upgrade. |
TCK 1.0-dev Compatibility Results (experimental)
|
Now that #36 (v1.0 wire format, PR 1 of #13) has landed on main, this harness will see a different set of "gaps" against Two things worth aligning on before this lands:
Concretely: rerun against current |
|
Also note that it's always good to run |
ITK Interoperability Harness + Baseline (showcase — no SDK changes)
This PR adds a test-harness that drives the unmodified A2A Elixir SDK against the official A2A Interoperability Test Kit (ITK), plus an honest capability/gap baseline report. It's a measuring stick, not a fix — zero changes to
lib/.Why
Establish a reproducible baseline so future v1.0-compliance work is gap-driven and regression-gated, and so reviewers can see exactly what the SDK can/can't do against a real A2A client today.
What's added (harness + docs only)
test/support/itk/instruction.ex— ITK Instruction protobuf codectest/support/itk/agent.ex— JSON-RPC handler / instruction interpretertest/itk/server.exs— standalone Bandit server (v0.3 card, JSON-RPC, SSE)test/itk/{instruction,agent}_test.exs+ binary fixturesdocs/ITK_BASELINE.md— full capability/gap report with reproducible evidencegit diff --stat origin/main -- lib/is empty (verified).Baseline summary
✅ Works (404 unit tests green against pristine SDK):
preferredTransport: JSONRPC,protocolVersion: 0.3.0)return_response,steps,call_agent)message/sendround-trip❌ Gaps (vs
a2a-sdk 0.3.24Python client — documented, not fixed here):ROLE_AGENT/TASK_STATE_COMPLETED(proto-style); v0.3 client expectsagent/completedTaskStatusUpdateEvent/TaskArtifactUpdateEventunion (missingtaskId/kind)encode_agent_cardemits v1.0supportedInterfaces; v0.3 client wantspreferredTransport/additionalInterfacesPrioritized v1.0 gap list (seeds future, separate PRs)
Reproduce